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TO COMPARE SEVERAL METH«:«BS C«F BEVELCipING CLASSR«:»>i 
QUESTIC-NING (FRC6ING) TECHNIQUES VIA DISTRIBUTED PRACTICE AND 
IMMEDIATE FEEDBACK, WEN THE LATTER EMPLOYED VIDEOTAPED 
FERFC*RMANCES «DF the LEARNER, 85 INTERNS WERE VIDEOTAPED C»N 4 
CCCASIONS DURING- THE FIRST 20 MINUTES Cf REGULAR CLASSRODM 
LESSC«NS. IN BETV/EEN TARINGS THEY RECEIVED 30 MINUTES CF 
SUPER VISI«DN, IN WHICH THEY VIEWED PLAYBACKS Cf EARLIER 
TEACHING ALCi%G WITH A CRITIQUE FRC>M AN EXPERIMENTER WHD 
PROVIDED DISCRIMINATION TRAINING. WITHIN-SESSIC«N FEEDBACK WAS 
HELD CONSTANT, AND AWrNrJT «DF PRACTICE AND DELAYED FEEDBACK 
WAS MANIPULATED, OVER 4 EXPERIMENTAL GRCOPS. A R3ST-TEST WAS 
VIDEOTAPED ABOUT 7 WEEKS AFTER TRAINING. INTERNS V/ERE TRAINED 
IN FRCeiNG TECHNIQUES (CLARIFICATION, CRITICAL AWARENESS, 
REDIRECTION, PRONFTING, REFOCUS) WHICH DEPENDED ON PUPIL 
RESPONSE, AS WELL AS AN ENOjURAGING DIVERGENT THINKING, ROLE 
FLAY IN BRIEF, AND PUPIL SUMMARY. TREATMENT DIFFERENCES, 
THOUGH NOT ENTIRELY CONSISTENT, FAVORED MASSED 
PRACTICE-IMMEDIATE FEEDBACK OVER DISTRIBUTED 
PRACTICE-REINSTATED FEEDBACK IN INITIAL ACQUISITION CF 
PROBING BEHAVIORS. THE FORMER ALSO* PRODUCED SIGNIFICANTLY 
MORE FREQUENT FR«:6ING THAN DISTRIBUTED PRACTICE AND IMMEDIATE 
FEEDBACK. RETENTION I^FERENCES CAN BE DRAW FRON THE FACT 
THAT DISTRIBUTED PRACTICE-DELAYED FEEDBACK GROBPS MAINTAINED 
HIGHER PROBING RESPONSE RATES ON THE F*:6T-TEST THAN DID 
MASSED PRACTICE-IMMEDIATE FEEDBACK. (AF) 
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EFFECTS OF FEEDBACK AND PRACTICE CONDITIONS 
ON THE ACQUISITION OF A TEACHING STRATEGY 



A key problem in the development of instructionat systems designed to produce 
teaching skiUs is that of providing adequate feedback on tlie teacher's performance. Quite 
apart from die practical limitations encountered in trying to schedule immediate feedback 
sessions with teacher trainees, supervision tends to focus on the end products of performance 
rather than the course of learning. This if? simply due to the fact that it is not usually 
possible to provide feedback during the teaching performance. Discrimination training is 
necessarily based on the perceptions of the teacher and the supervisor as they remember 
the lesson. The entire process thus tends to invite a heavy reliance on private frames of 
reference. The supervisor and the teacher do not start from a common perception of vhat 
was done, how it was done, and what the effects were ^ 

Perhaps the signal advantage of televising trainee lessens is that in subsequent super- 
vision sessions, the original performance can be completely reinstated. In this way, die 
teacher intern is not required to respond to supervision on the basis of ^at he and the 
supervisor recall about a complex series of events. Further, the usual inhibitory effects 
cf delayed feedback may be offset, not only because the initiid teaching performance is 
reinstated; bitt because it is possible to provide discrimination training at any point in 
the development of a response sequence. Rather than reacting to performance in terms cf 
end products, it is possible for die intern and supervisor to analyze a given interaction 
sequence in die classroom, identify salient cues, and develop strategies to improve further 
performance. The results of an earlier experiment by McDonald, Allen, and Orme (1966) 

V 

support diese hypotheses. They found that the provision of discrimination training during 

% 

videotaped playbacks of intern lessons produced significantly greater increases in selected 
teacher behaviors than did confirmation or self-feedback procedures. 

If the above reasoning is valid, and the reinstatement of original performance by 
videotape does in fact solve the problem produced by delaying feedback, die problems con- 
cerning the optimal arrangement of feedback and interspersed practice sessions take on new 
meaning. For example, given die capability to reinstate the original performance by video- 
tape at any point in time, should practice precede or follow supervision? In the first case, 
the acquisition of a complex skill mig^t be slower. However, substantial gains might be 
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realized because the learner becomes less defensive about feedback based on his earlier 
performance} and thus becomes more responsive to supervision* Tliis would be ’’delayed** 
feedback in die sense that die learner was receiving training based on an earlier performance* 
In this kind of a training situation} the learner would be serving as his own model* 

IVhere supervision immediately followed performance} the learner’s e^-involvement 
in the lesson just taug^ might lead him to be less open to suggested change* It is also 
probable} of course} that the immediacy of supervision would outweigh such potential 
resistance* 

Ihere is a general consensus diat initially at least, massed practice is optimal for 
the acquisidon of comfflex skills and reascming strategies* Given dme limitadons in trailing, 
however, we are sdll left widi die problem of retendon* McDonald, ^Uen, and Orme (1966) 
found that ^diile interns quickly reached criterion (vhere die dependent variable was amount 
of reinforcing of pupil pardcipadon) inidal gains were not lasting* Ihis result suggests 
that distributed pracdce would be a logical altemadve* Further, given die perceptual 
adequacy of videotaped feedback, it is quite possible that acquisidon rates would not be 
signidcandy lo^r dian under massed pracdce condidons* 

Finally, idiile there is considerable evidence to sugge^ that distributed pracdce is 
generally superior to massed pracdce, idiy diis is so is not known* Recent research by 
McCaugh and Hostetter (1961) tends to support a consolidadon theory of learning. However, 
as Hilgard (1962) points out:, none of the current theories account for excepdons to the 
general rule* In training, t/’ien, the opdmal organizadon of pracdce periods becomes an 
inelegant process of empiriccUly establishing die limits of each approach in terms of a 
given skill or strategy* The i^xirpose of the experiment reported here was to compare; 
several methods of distributing; pracdce and feedback vhen the latter employed videotaped 
performances of the learner* 

General Procedure: The general procedures followed in this research were similar to diose 
reported in the first experiment (McDonald, Allen, and Orme, 1966) in a series of feeback 
studies* Mem teachers were vfdeotaped on four separate occasions, during the first 20 
minutes of regular classroom lessons* In the intervals between each of the taping sessions, 
they received experimental tre jtments within die context of regular supervision* 












The treatment of supervision sessions were sliloe for all subjects in that they viewed 
videotaped pla 3 d>acks of tiieir earlier teaching performance with an experimenter who pro- 
vided discrimination training. However, instead of varying the amount of feedback within 
each session as in the first experiment, we held within-session feedback constant, and 
manipulated the amount of practice and delayed feedback over four experimental groups. 

In addition, tibe post-test was videotaped approxiinately seven weeks after training so as 
to permit inferences about retentiQn as well as acquisition curves. 

The Dependent Variable: hi the earlier experiment, interns had been trained to reinforce 
pupil participation in discussion and review lessons. / natural extension of this skill 
seemed to be tiiose techniques where die teacher could increase the quality of such par- 
ticipation. 

Radier than attempting to operationally define a ''penetrating question, " or to develop 
techniques for suppressing superficial "first-answer" pupil responses, the approach taken 
was to dev^op classroom techniques which followed simple shaping procedures. Interns 
were given three basic rules: (1) Do not give immediate answers to pupil questions. (2) 
Once a pupil has responded, try to get him to "go beyond" the information given on^ 
several specified techniques). (3) Differentially reinforce pupil responses diat demonstrate 
Increased critical awareness. 

In the initial written instructions to eadi intern, and in supervision as well, a series 
of discrete techniques were presented following die statement of each rule. The basic 
strategy in each case was the same. Following a pupil response, the teacher asked a 
question designed to elicit more information or more meaning from the pupil. If the pupil 
response was adequate or insig^thil, the teacher dien attempted to get him to relate the 
questions, answer, or comment to another area, or to spell out its implications for a given 
Issue. 

These various techniques were presented as exemplars of a basic questioning 
strategy termed probing. Specific techniques were defined as: clarification , critical 
awareness , redirection, pro.^pting and refocus. (The term used to describe each specific 
category reflects die teacher's goal vhen using a given technique. ) 

hi addition, a secondary set of techniques designed to achieve the same ends as 
probing procedures were included as part of the training problem. 



■n« latter techaidues differ from ptobtag in teat they may or may not depend on a 

prior pupU response, l.e., Oe teacher can Introduce tihem at any point In the discussion 

himself, or use them as specific types of probes, these techniques «ere termed encouraging 

dternattjMdlvergem telnking). supposition (role play in brief) and aiasummarylT^"” 

cusaion. Both in training and in later tape analysis, the popU response served as an Sd to 

cue the intern or rater that a teacher probing (classifiable) response should Immediately 
follow, ' 



Treatment Conditions: Stanford interns were assigned to one of four experimental groups, 

eac "f’dilch received varying amounts of practice betv«en feedback sesslo^^ Subjects in 

each group received dlscrlmlnatloo training from one of two Es eho were present at each 

feedback session. AU feedback sessltms were thfrty minutes In length. A summary of the 

design appears in Table I. (See Table 1, Experiment ff). 

Group 1, (immediate Feedback Group , life those in the 

other diree groups, initially received written Instructions on probing. The instruction 
related to new techniques to the interns*, prior training in reinforcement, and stated that fee 
aslcjam Of the current study was to help Interns develop a broader range of basic questlon- 
s in discussion and review lesscms. Each rule was then stated, and specific tech- 
niques together with relevant examides were presented. One of fee two E* then briefly 
reviewed the techniques wife the Intern, reinforced positive statements about fee potential 
3_Uty of probing techniques, and viewed fee Intern's playback wife him. 

Each time a pupU responded verbaUy, E cued the Intern, and if fee latter had probed 

^en E reinforced him. « he had not, E suggested how this might be done, ihtemslnthe 

rst group were both videotaped and supervised on three consecutive days. They were 

^deotaped at the school on a given day, and received supervision based on that performance 

e same evening. The post-test was videotaped 45 days after the pretest was made. 

_ roup 2 (teunedlate Feedback - Distributed Practlcel: These subjects received fee a»i~> 

Und of discrimination training as those In Group 1. Instead of receiving massed practice 

however, they were directed to practice probing techniques for a one-week period following 
each of fee supervision sessions. 

Supervision sessions were based on videotapes that had been recorded earUer on fee 
same day. The Post-test foUowed fee pretest videotape by 45 days. 
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Group 3 (Delayed Feedback Distributed Practice): Subjects in this group received the same 
supervision treatment as those in Group 2. However, tiie taping schedule was arranged in 
such a way that they always viewed a videotape of their performance which had been filmed 
one week earlier. This has been termed delayed feedback since unlike Group 1 and 2 sub- 
jects who received feedback based on that day’s performance, these subjects were taped on 
the day following supervision, and received no discrimination training based on their per- 
formance for one week. At the conclusion of each session Group 3 subjects were directed to 
practice probing techniques not only in the next day's lesson, but during the rest of the wed; 
as well. 

G roup 4 (Reinstated Feedback - Distributed Practice); Hiese subjects received treatment 
distributed over a six-week period. Following taping on the first day of week one, they 
received supervision based on that tape on the first day of week two. Throughout treatment, 
they were taped and supervised on alternate weeks. As was the case with the first three 
groups, the fourtii group were post -tested 45 days after the pretest had been filmed. 

Subjects: Interns were selected from the same population as that for the first experiment, 

A total of 85 intern teachers majoring in English, Social studies. Mathematics or Science 
were selected for study. Approximately equal numbers of interns from each subject-matter 
area, and from socio -economically equivalent schools were assigned to each of the four 
groups. Mean age for the four groups varied from 23,7 to 25,5 years. Sex differences and 
subject -matter major in each group were very similar to those already described in the 
first experiment. 

Me asurement procedures; Four videotapes for each intern in each group were analyzed by 
raters trained for this pmrpose. Four raters received intensive training on non-experimenlal 
tapes before the analysis of the latter was begun., Training consisted of joint rating sessions- 
that continued until there was 95% agreement on the major response categories. 

Reliability was maintained by frequent analyses of double rated tapes® In addition, 
;!oint rating sessions were held after each block of 50 tapes had been completed so that 
systematic rater biases could be forestalled. As rating progressed, it was found that cer- 
tain tapes inevitably yielded low interrater agreement. This occurred Mien the intern in 
question was unable to maintain classroom discipline, or Mien there was an unusually high 

^ There was a three -mondi break between the conclusion of the first study and the beginning 
of the study being described here. 
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rate of Interaction combined with "fuzzy*' audio or visual output. In these cases, the tapes 
were rated by Independent teams of raters by dividing the rating task between themselves 
and replaying frequently, two raters were able to record all Interactions. To control for 
combined rating elfects, independent teams were used. The Identlflcatloa of these "trouble*' i 
tapes posed a problem, since when raters were asked to rank the rating difficulty of tapes 
their Judgments were not entirely consistent reliability checks. For this reason. It was 
finally decided to double«rate all tapes. 

The results of diese control procedures are reflected in the reliability coefficients 
reported In Table 2. (See Table 2, Experiment II) The videotapes on ^ch they are based 
were selected in the following manner. Since coefficients based on all of the tapes would 
have been prohibitively expensive and also tlnie*consumlng, 160 tapes were selected using 
tables of random numbers. Five tapes from each group for each trial were randomly 
selected Irtnn pools of 15 to 21 tapes. The coefficients reported then are based on a 
representative aelectiop of Independently rated tapes from each group and for each trial. 

As can be seen, Inicerrater agreement’ on the major response categories Is very high. 

A certain amount of data were lost between Initial videotaping and statistical analysisc 
Forty-four tapes were omitted at file outset because of technical Inadequacy or unduly short 
tape time. Flfty-slx tiqies were prorated to bring fiiem up to file 20.0 minute criterion. 

Tapes less fiian 15 minutes In length were omitted, those between IS and 20 minutes were 
prorated. Before any of the above adjustments were made, the T statistic was applied to 
mean tape time and number of omitted tapes per cell to determine whether or not any 
significant differences between cells existed. The results were well short of significance as 
these tapes were almost equally divided among all cells. The standard errors for short 
tapes varied from .40 to 1 .2 minutes. As a further check on file data, the mean number of 
days between pre and post-test were calculated for each experimental group. The overall 
mean was 45.25 days, with 1.41 days. 

RESULTS 

Probing was presented as a basic que.stioning strategy in training, and specific 
techniques such as Refocus or Critical awareness were subsumed under this broad general 
concept. It is important to note, however, that this was a logical distinction made to 

2 

Team ratings were not Included in selection because they had been rated by special 
techniques, and were known to be highly reliable on all major response categories. 




facilitate training. It does not follow that it is psychometrlcally relevant* In fact, the 1 

analysis of results did not proceed in terms of o*ie broad dependent variable, but in terms | 

of eig^t. Each of the questioning and related techniques were viewed as discrete dependent j 

variables. Table ,3 provides support for this procedure. The Intercorrelations between i 

the subcategories of probing and non-probing are very low. Out of the 78 relevant coefficients, | 
only 5 are significantly different from zero. Of these, 3 reach the .05 level, and 2 reach J 
the .01 level. Prompting is significantly related to redirect (ir = .23, p = .05), refocus j 

(r = .28, p = .05) and clarification (r = .32, p = .01). j 



However, prompting occurred infrequently (group means varied from 3.7 to 1.7) 
and contributed little to overall differences. 

In the analysis of treatment differences then, F Ratios for aU of the major probing 
techniques must be considered. F ratios based on probing provide a general overview of ms 

results taken as a whole, but since they include variables which did not reflect significant 
differences, there is a suppression effect. 

Treatment Differences: Analyses of covariance with relevant trial 1 scores as covariates 
were carried out to determine between-group differences in trials 2, 3, and 4. Table 4 

summarizes these results. As can be seen, differences are reflected in several of the 
dependent variables. ^ 

Treatment differences were most prominent in trial 3. The exoerimental groups 
were found to differ significantly from each other in the use of critical awareness (o^.Ol). 
redirection (pc05), refocus (p cOl) and role play (p<.05) however, the relative frequency 
of occurrence for each of the response categories that determine treatment differences is 
important in attaching meaning to these results. For example, role play occurs Infrequently 
and was slgniffcant, viille clarification which contributes heavily to total teacher responses 
was not. This is reflected in the rather low F ratio for probing, which while it suggested a 
strong trend, was not significant (p d.250). 

Treatment differences In trials 2 and 4 are much less general. In trial 1, redirection 
was slgniffcant (p <.05) and clarification , refocus and pupil summary approach significance, 
but within -group variance is great enough to reduce the F ratio to 0.600, for probing.. By 
trial 4 only refocus shows a clear trend (p<.10) for treatment effects. 

In general, die most frequently occurring responses are listed first in the tables. Further^ 
an Inspection of Table 3 shows that the more frequently a response occurs, the more highly 
correlated It is with probing. " 
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An overall summary of treatment differences derived from the analyses of covariance 
was shown in Table 5* (See Table 5, Experiment II) Adjusted means, standard errors and 
F ratios for mean frequencies of probes by groups over trials are presented. Figures 1 and 
2 (See Figures 1 and 2, Experiment II) illustrate the general pattern of probing, and clari^v 
specific between-group differences for a given trial. Both figures are based on unadjusted 
means so that trial 1 levels of performance can be seen in relation to the other three trials 
Id. Figure 1, all unadjusted probing means were brought to a common point somewhat 
analogous to a covariance adjustment by dividing the respective means for trial one into each 
of the four means for each group. This clearly illustrates specific between-group differen- 
ces vblch also appear in Table 6. (See Tabel 6, Experiment H) 

In trial 2,Groups 1 and 4 differed significantly (p<.05) from Group 2. Group 3 
subjects also tended (p<.10) to redirect more frequently than did Group 2 subjects. In 
trial 3, Groups 1 and 3 interns used redirection, refocus and critical awareness significant- 
ly more often than did Group 2 and Group 4 subjects (Table 6). 

Figure 1 tends to exaggerate certain group trends, particularly from trials 3 to 4. 
Figure 2 is singularly instructive here because it considers all of the dependent variable in 
relation to total teacher responses. In this bigure plot points were derived by expressing 
probes as a percentage of total teacher responses. Now, comparing hgure 1 with figure 2, 
it can be seen that Group 3 and 4 do not drop off from trial 3 to trial 4 as implied in figure 1 . 
The relevant conclusion then is that the major treatment differences still lie in trial 3, but 
that retention rates for Groups 2, 3, and 4 are considerably higher than a consideration of 
probes in isolation to other responses would indicate. 

Training Differences ; Table 7 (See Table 7, .Experiment II) summarizes the training 
or within-group differences for the four groups. . The probing category, effectively represents 
these differences. Group 1 subjects realized their greatest gains in the discrete probing 
techniques by trial 3. This was also true for Group 3 and 4 subjects. However, as 
Figure 2 illustrates, the latter groups did not drop off as extensively did Group 1 in later 
trials. 
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Discussion of Results 

Treatment differences were not entirely consistent throughout trials and across 
dependent variables, so that conclusions must be qualified. It is clear however that there 
v/ere significant treatment differences in favor of Group 1 (Massed Practice and Immediate 
feedback) over Group 4 (Distributed Practice and Reinstated Feedback) in the initial 
acquisition o f probing behaviors. This occurred by trial 3. 

Group subjects also probed significantly more frequently than Group 2 (Distributed 
Practice and Immediate Feedback) subjects, although differences between these latter 
two groups may also be influenced to some extent by differences in the temporal contiguity 
between supervision and subsequent videotaping. Group 1 subjects were taped immediately 
after supervision, while Group 2 subjects were exposed to a one week practice period 
before tiiey were videotaped the next time. 

Certain inferences can also be drawn about treatment differences related to retention. 
Figure 2 indicates that the distributed practice and delayed feedback groups (Groups 2 and 
4) maintained relatively higher probing response rates on the post-test than did Group 
1 who dropped off quite sharply. However, the data do not permit firm conclusions here 
as F ratios for trial 4 were not significant. 

Finally, it should be pointed out that there appear to be practical limits on the 
amount of probing possible in any given class period . Unlike teacher reinforcement 
which may occur in high frequency, classroom and subject matter concerns establish a 
ceiling for probing, and this is why the most sensitive measures of this variable are 
likely to be expressed as ratios or percentages related to total teacher responses. 
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TABLE 2 
(Exp. n) 



RELIABILITY COEFFICIENTS BASED ON A RANDOMIZED SAMPLE OF 80 
VIDEOTAPES RATED BY TV/O INDEPENDENT OBSERVERS 



Response Category 


Reliability Coefficient 


Total Pupil Responses 


0.9934 


Total Teacher Probes 


0.9939 


Total Teacher Non Probes 


0.9871 


Total Teacher Reinforcement 


0.9579 


Probing Sub-Categories: 




a. Clarification 


0.9689 


b. Critical Awareness 


0.8658 


c. Redirect 


0.7633 


d. Prompting 


0.5914 


e. Refocus 


0.7312 

1 1 




TABLE 3 (Exp. H) 



CORRELATION MATRIX FOR THE MAJOR RESPONSE CATEGORIES 
I OF THE DEPENDENT VARIABLE 

I N = 79 
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3TE: In order for any of the coefficients reported above to be considered significantly different 
from o, r must = ,232 (p = .05) or .303 (p = .01). 

N. of Correlation Coefficients = 78 
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TABLE 4 
(Exp. n) 



SUMMARY OF F RATIOS AND SIGNFICANCE LEVELS FOR THE 
DEPENDENT VARIABLES, DERIVED FROM TRIAL TWO, THREE AND FOUR 
ANALYSES OF COVAi^IANCE, WITH TRUL ONE SCORES AS COVARIANTS 




TABLES 
(Exp. n) 

SUMMARY OF THE ANALYSES OF COVARIANCE FOR TliE EXPERIMENTAL 
GROUK, WITH TRIAL ONE SCORES AS COVARIANTS AND RPOBING 

AS THE DEPENDENT VARIABLE 
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Level cf significance for a; F (3/69) = 1.41, p = .250 
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TREATMENT DIFFERENCES BETWEEN GROUPS IN TRIAL TWO AI^ TOREE 

BASED ON THE T STATISTIC 



Response Category 



Trial 2 



Trial 3 



Direction of Difference Between Groups Within a Given Trial 



Redirection 


F Ratio 


} F Ratio 
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1 
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cGj (p <.0S) 
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1 
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G^ < Gj - .05) 

G^< Gg ftx.lO) 

Gg< Gg (p c .10) 
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G^ ^ Gj^ (p 'i^.lO) 
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TABLE? 

(Exp.n) 

TRAINING DIFFERENCES FOR THE FOUR EXPERIMENTAL GROUPS FROM 
TRIAL TO TRIAL, ON IvlAJOR RESPONSE CATEGORIES 



Response Category 


! 1 i 

Group 1 j Group 2 | Group 3 \ Group 4 


Direction of Difference from Trial to Trial and p level (two-tailed) 


Total Pupil Responses 


NS 


i 


1 

1 

T, < Tg (.05) 
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NS 
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NS 


NS 


NS 
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NS 


NS 


NS 
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NS 
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o 



V 



(Exp, n) 



1.60 



1.55 



1.50 



1.45 



1.40 



1.35 



1.30 



1.25 



1.20 



1.15 



1.10 




Group 1 (C) 



- Group4(R4D) 






Group 2 (S-F) 



Group 3 (R) 



Teaching Sessions 

FIGURE 1. Unadjusted mean frequencies for probing brought to a common point by using 
trial one means as divisions in each trial. 
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FIGURE 2. Percent of prc^s contributing to total teacher responses for all groups 
over all trials. 
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Teaching Sessions 

FIGURE 2. VexcevLt of probes contributing to total teacher responses for all groups 
over all trials. 
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